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REMARKS 

Claims 10-16 are presented for consideration, with Claims 10, 15 and 16 being 
independent. The independent claims have been amended to further clarify the features of 
Applicants' invention. Support for the claim amendments can be found, for example, on page 
12, line 12, et seq., of the specification and element 506 of Figure 5. As such, no new matter has 
been added. 

Claims 10 and 13-16 stand rejected under 35 U.S.C. §103 as allegedly being obvious 
over Williamson et al. (U.S. Patent Application Publication No. 2004/0104935) in view of 
Latham (U.S. Patent No. 5,803,738). In addition, Claims 1 1 and 12 are rejected as allegedly 
being obvious over those citations and further in view of the Lescinsky publication ("Interactive 
Scene Manipulation in the Virtue3D System"). These rejections are respectfully traversed. 

Claim 10 of Applicants' invention relates to an image processing apparatus for 
compositing an image of a virtual object and an image of a physical space to generate a mixed 
reality image and causing an HMD to display the mixed reality image. The apparatus includes a 
database which holds 3D CG data and scene graph data, wherein the 3D CG rendering data 
contains data representing geographical shape and color of the virtual object, texture data, and 
data representing the position and orientation of the virtual object, and wherein the scene graph 
data is data in which the 3D CG rendering data is hierarchically managed on the basis of the 
parent-child relationship between components of the virtual object, an image capturing unit 
which is attached to the HMD and captures the image of the physical space, a first measurement 
unit which measures a position and orientation of the HMD, and an object manipulation unit 

-8- 



Appln. No.: 10/594,114 

which is used by a user wearing the HMD in order to operate a position and orientation of the 
virtual object. In addition, a second measurement unit measures a position and orientation of the 
object manipulation unit, an operation panel which can be operated by the user, arranged at a 
position in the physical space within a viewing field of said image capturing unit attached to the 
HMD, displays an operation panel image which is used for editing the virtual object, and 
contains a region for hierarchically displaying information about each of the components in 
accordance with the scene graph data held in the database, and is capable of receiving a user 
instruction of editing the virtual object input by the user, and an operation panel image generation 
unit generates the operation panel image by using the data held in the database and outputs the 
generated operation panel image to the operation panel. A rendering unit updates the data held in 
the database according to the user instruction received via the operation panel and the 
measurement result of the second measurement unit, and renders, by using the updated data, the 
image of the virtual object according to the measurement results of the first and second 
measuring units, and a composition unit composites the image of the rendered virtual object and 
the captured image of the physical space to generate the mixed reality image. Finally, a HMD 
displays the mixed reality image generated by the composition unit. 

Independent Claims 15 and 16 incorporate the features of Claim 10 but are directed to 
a method and a computer readable storage medium, respectively. The amended claims now 
recite, among other features, "a database which holds 3D CG data and scene graph data, wherein 
the 3D CG rendering data contains data representing geographical shape and color of the virtual 
object, texture data, and data representing the position and orientation of the virtual object, and 
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wherein the scene graph data is data in which the 3D CG rendering data is hierarchically 
managed on the basis of the parent-child relationship between components of the virtual object" 
and an operation panel that "displays an operation panel image which is used for editing the 
virtual object and contains a region for hierarchically displaying information about each of the 
components in accordance with the scene graph data held in the database, and is capable of 
receiving a user instruction of editing the virtual object input by the user." Applicants submit 
that none of the cited documents discloses these features. 

The Williamson et al. publication is relied on to teach, inter alia, a database that holds 
data used for generating the image of the virtual object (See page 3 of the Office Action) and an 
operational panel (See page 3 of the Office Action). The Williamson et al. publication, however, 
does not teach or suggest, the a database which holds 3D CG data and scene graph data, wherein 
the 3D CG rendering data contains data representing geographical shape and color of the virtual 
object, texture data, and data representing the position and orientation of the virtual object, and 
wherein the scene graph data is data in which the 3D CG rendering data is hierarchically 
managed on the basis of the parent-child relationship between components of the virtual object. 
The final Office Action states, on page 3, that the Williamson et al. publication does not teach, 
among other things, an operation panel which can be operated by a user, arranged at a position in 
the physical space within a viewing field of an image capturing unit attached to a HMD. 
Applicants further submit that the Williamson et al. publication does not teach or suggest the 
operational panel that displays an operation panel image which is used for editing the virtual 
object and contains a region for hierarchically displaying information about each of the 
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components in accordance with the scene graph data held in the database, and is capable of 
receiving a user instruction of editing the virtual object input by the user, as set forth in amended 
Claim 10. 

The Latham patent is relied on to teach, on page 4 of the final Office Action, an 
operation panel which can be operated by a user. In the Advisory Action, the Examiner states 
that Latham presents a physical control panel in the physical space with is view by the user via 
the HMD image capture device, referring to Fig. 1. Latham teaches a system for simulating the 
forces associated with touching objects in a virtual reality simulator system. Applicant submits, 
however, that Latham does not teach a operation panel, which displays an operation panel image 
which is used for editing a virtual object. In Figure 1, Latham depicts a image 10 of an operation 
panel to a user wearing a HMD 20. This image 10, however, is not presented on an operation 
panel that his part of the physical space. Further, image 10 is not an operation panel image which 
is used for editing a virtual object. An panel 12 is positioned to mirror the movement on the 
hand. This panel 12 does not include a display panel for displaying operation panel image which 
is used for editing a virtual object, and cannot be arranged at a position in the physical space 
within a viewing field of an image capturing unit attached to a HMD, as Latham does not teach 
an image capturing unit attached to a HMD. As such, Applicants submit that the Latham does 
not teach or suggest an operation panel, which displays an operation panel image which is used 
for editing a virtual object, arranged at a position in the physical space within a viewing field of 
an image capturing unit attached to a HMD, as previously set forth in Claim 10. 
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Further, the Latham patent also fails to overcome the deficiencies of the Williamson et 
al. publication discussed above, with regards to amended Claim 10. That is, in contrast to 
Applicants' claimed invention, Latham does not teach or suggest, among other features, a 
database holding the claimed data, along with the operation panel that displays the claimed 
operational panel image, as recited in Claim 10. 

Accordingly, without conceding the propriety of combining Williamson et al. and 
Latham , such a combination still fails to teach or suggest Applicants' claimed invention. The 
tertiary citation to the Lescinsky publication relates to interactive scene manipulation and was 
relied on for teaching the generation of a image of a virtual object based on 3D CAD data. The 
Lescinsky publication fails, however, to compensate for the deficiencies in Williamson et al. and 
Latham as discussed above. 

Claims 15 and 16 relate to an image processing method and a computer readable 
storage medium, respectively, and correspond to Claim 10. These claims are thus also submitted 
to be patentable over the art discussed above. 

For the foregoing reasons, Applicant respectfully submits that each of independent 
claims 10, 15 and 16 is patentable over the applied art of record. In addition, dependent Claims 
11-14 set forth additional features of Applicants' invention. Independent consideration of the 
dependent claims is respectfully requested. Therefore, reconsideration and withdrawal of the 
rejection of Claims 10 and 13-16 under 35 U.S.C. §103 is respectfully requested. 

Applicant respectfully submits that all outstanding matters in the above application 
have been addressed and that this application is in condition for allowance. Favorable 
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reconsideration and early passage to issue of the above application is respectfully sought. 

Applicants' undersigned attorney may be reached in our Washington, D.C. office by 
telephone at (202) 530-1010. All correspondence should continue to be directed to our below- 
listed address. 

Respectfully submitted, 



/Lawrence A. Stahl/ 



Lawrence A. Stahl 
Attorney for Applicants 
Registration No. 30,110 

FITZPATRICK, CELLA, HARPER & SCINTO 
1290 Avenue of the Americas, 17 th Floor 
New York, NY 10112-3800 
Facsimile: (212) 218-2200 
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